A potentially more useful na.action setting in lm is na.exclude which pads out predicted values and residuals with NAs instead of omitting the 30 observations listed above.
Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls
In this article, we review the reasons why missing data may lead to bias and loss of information in epidemiological and clinical research. We discuss the circumstances in which multiple imputation may help by reducing bias or increasing precision, as well as describing potential pitfalls in its application. Finally, we describe the recent use and reporting of analyses using multiple imputation in general medical journals, and suggest guidelines for the conduct and reporting of such analyses.
https://www.bmj.com/content/338/bmj.b2393
Note: The next 7 slides are derived from Sterne et al.
An Example from Sterne et al.
Consider, for example, a study investigating the association of systolic blood pressure with the risk of subsequent coronary heart disease, in which data on systolic blood pressure are missing for some people.
The probability that systolic blood pressure is missing is likely to:
decrease with age (doctors are more likely to measure it in older people),
decrease with increasing body mass index, and
decrease with history of smoking (doctors are more likely to measure it in people with heart disease risk factors or comorbidities).
If we assume that data are missing at random and that we have systolic blood pressure data on a representative sample of individuals within strata of age, smoking, body mass index, and coronary heart disease, then we can use multiple imputation to estimate the overall association between systolic blood pressure and coronary heart disease.
Missing Data Mechanisms
Missing completely at random There are no systematic differences between the missing values and the observed values.
For example, blood pressure measurements may be missing because of breakdown of an automatic sphygmomanometer.
Missing at random Any systematic difference between the missing and observed values can be explained by other observed data.
For example, missing BP measurements may be lower than measured BPs but only because younger people more often have a missing BP.
Missing not at random Even after the observed data are taken into account, systematic differences remain between the missing values and the observed values.
For example, people with high BP may be more likely to have headaches that cause them to miss clinic appointments.
“Missing at random” is an assumption that justifies the analysis, and is not a property of the data.
Trouble: Data missing not at random
Sometimes, it is impossible to account for systematic differences between missing and observed values using the available data.
In such (MNAR) cases, multiple imputation may give misleading results.
Those results can be either more or less misleading than a complete case analysis.
For example, consider a study investigating predictors of depression.
If individuals are more likely to miss appointments because they are depressed on the day of the appointment, then it may be impossible to make the MAR assumption plausible, even if a large number of variables is included in the imputation model.
Where complete cases and multiple imputation analyses give different results, the analyst should attempt to understand why, and this should be reported in publications.
What if the data are MCAR?
If we assume data are MAR, then unbiased and statistically more powerful analyses (compared with analyses based on complete cases) can generally be done by including individuals with incomplete data.
There are circumstances in which analyses of complete cases will not lead to bias.
Missing data in predictor variables do not cause bias in analyses of complete cases if the reasons for the missing data are unrelated to the outcome.
In such cases, imputing missing data may lessen the loss of precision and power resulting from exclusion of individuals with incomplete predictor variables but are not required in order to avoid bias.
Stages of Multiple Imputation (1 of 2)
Multiple imputation … aims to allow for the uncertainty about the missing data by creating several different plausible imputed data sets and appropriately combining results obtained from each of them.
The first stage is to create multiple copies of the dataset, with the missing values replaced by imputed values.
The imputation procedure must fully account for all uncertainty in predicting the missing values by injecting appropriate variability into the multiple imputed values; we can never know the true values of the missing data.
Note that single Imputation of missing values usually causes standard errors to be too small, since it fails to account for the fact that we are uncertain about the missing values.
Stages of Multiple Imputation (2 of 2)
The second stage is to use standard statistical methods to fit the model of interest to each of the imputed datasets.
Estimated associations in each of the imputed datasets will differ because of the variation introduced in the imputation of the missing values, and they are only useful when averaged together to give overall estimated associations.
Standard errors are calculated using Rubin’s rules, which take account of the variability in results between the imputed datasets, reflecting the uncertainty associated with the missing values.
Valid inferences are obtained because we are averaging over the distribution of the missing data given the observed data.
Comparing Two Linear Models including Multiple Imputation
See https://www.framinghamheartstudy.org/ for more details.
Codebook for Today
Variable
Description
educ
four-level factor: educational attainment
smoker
1 = current smoker at examination time, else 0
sbp
systolic blood pressure (mm Hg)
obese
1 if subject’s bmi is 30 or higher, else 0
glucose
blood glucose level in mg/dl
The variables describe adult subjects who were examined at baseline and then followed for ten years to see if they developed incident coronary heart disease during that time.
Use linear regression to predict sbp using two different models, in each case accounting for missingness via multiple imputation, where the predictors of interest are glucose, obese, educ, and smoker.
Residual standard error: 1.14 on 3833 degrees of freedom
(402 observations deleted due to missingness)
Multiple R-squared: 0.05531, Adjusted R-squared: 0.05481
F-statistic: 112.2 on 2 and 3833 DF, p-value: < 2.2e-16
Residual standard error: 1.126 on 3733 degrees of freedom
(498 observations deleted due to missingness)
Multiple R-squared: 0.07919, Adjusted R-squared: 0.07771
F-statistic: 53.5 on 6 and 3733 DF, p-value: < 2.2e-16
but there are cart methods and many others available, too.
What should we include in an imputation model?
If things you are imputing are not Normally distributed, this can pose special challenges, and either a transformation or choosing an imputation method which is robust to these concerns is helpful.
Include the outcome when imputing predictors. It causes you to conclude the relationship is weaker than it actually is, if you don’t.
The MAR assumption may only be reasonable when a certain variable is included in the model.
As a result, it’s usually a good idea to include as wide a range of variables in imputation models as possible. The concerns we’d have about parsimony in outcome models don’t apply here.
Store one (or more) of the imputed data sets
This will store the fifth imputed data set in imp_5.
par(mfrow =c(1,2))plot(m4_mods$analyses[[6]], which =c(1:2))
Residual Plots for mod4 (6th imputation)
par(mfrow =c(1,2))plot(m4_mods$analyses[[6]], which =c(3,5))
par(mfrow =c(1,1))
Residual Plots for mod4 (1st imputation)
par(mfrow =c(1,2))plot(m4_mods$analyses[[1]], which =c(1:2))
Residual Plots for mod4 (1st imputation)
par(mfrow =c(1,2))plot(m4_mods$analyses[[1]], which =c(3,5))
par(mfrow =c(1,1))
Guidelines for Reporting
Guidelines for reporting, I (Sterne et al.)
How should we report on analyses potentially affected by missing data?
Report the number of missing values for each variable of interest, or the number of cases with complete data for each important component of the analysis. Give reasons for missing values if possible, and indicate how many individuals were excluded because of missing data when reporting the flow of participants through the study. If possible, describe reasons for missing data in terms of other variables (rather than just reporting a universal reason such as treatment failure.)
Clarify whether there are important differences between individuals with complete and incomplete data, for example, by providing a table comparing the distributions of key exposure and outcome variables in these different groups
Describe the type of analysis used to account for missing data (eg, multiple imputation), and the assumptions that were made (eg, missing at random)
Guidelines for reporting, II (Sterne et al.)
How should we report on analyses that involve multiple imputation?
Provide details of the imputation modeling (software used, key settings, number of imputed datasets, variables included in imputation procedure, etc.)
If a large fraction of the data is imputed, compare observed and imputed values.
Where possible, provide results from analyses restricted to complete cases, for comparison with results based on multiple imputation. If there are important differences between the results, suggest explanations.
It is also desirable to investigate the robustness of key inferences to possible departures from the missing at random assumption, by assuming a range of missing not at random mechanisms in sensitivity analyses.
